Skip to content

feat(hosts): add pi coding agent adapter - #675

Open
wutongyuonce wants to merge 10 commits into
NevaMind-AI:mainfrom
wutongyuonce:feat/pi-host-adapter
Open

feat(hosts): add pi coding agent adapter#675
wutongyuonce wants to merge 10 commits into
NevaMind-AI:mainfrom
wutongyuonce:feat/pi-host-adapter

Conversation

@wutongyuonce

@wutongyuonce wutongyuonce commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

📝 Pull Request Summary

Add a dedicated memu-pi adapter for pi coding agent's v3 JSONL sessions,
global instructions, skills, and scheduled bridging.

Fixes #674.


✅ What does this PR do?

  • adds PiTranscriptSource for ~/.pi/agent/sessions/<encoded-cwd>/*.jsonl;
  • keeps user/assistant text in the conversation track while routing
    toolResult, bashExecution, and pure assistant toolCall entries to the
    tool track;
  • adds the memu-pi CLI by reusing the existing HostSpec and bridging
    pipeline;
  • supports PI_CODING_AGENT_DIR and PI_CODING_AGENT_SESSION_DIR;
  • adds macOS/Linux cron and Windows Task Scheduler installation guidance;
  • redirects memu-agent detect ~/.pi to the dedicated adapter;
  • updates the adapter tables and adds focused classifier/scheduling tests.

🤔 Why is this change needed?

The generic typed-tree classifier treats pure assistant toolCall rows as
conversation messages. A read-only sample of 10 recent real pi sessions found
385 such rows. The dedicated source classifies all 385 as tool records and can
also use pi's stable instruction, skill, session, and headless execution paths.


🔍 Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor / cleanup
  • Other (please explain)

✅ PR Quality Checklist

  • PR title follows an allowed format
  • Changes are limited in scope and easy to review
  • Documentation updated where applicable
  • No breaking changes
  • Related issues or discussions linked

Validation:

  • make check
  • make test (604 passed, 8 skipped)
  • real-session classifier check: 385/385 pure tool-call rows classified as
    TOOL

📌 Optional

  • Screenshots or examples added (not applicable)
  • Edge cases considered
  • Follow-up tasks mentioned

@xnne-bot

xnne-bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for adding a new host adapter. New-host reviews cover several independent seams — installation, scheduler lifecycle and real wake-up behavior, transcript preparation/shape, self-session exclusion, retrieval injection, and cross-platform cleanup — so this review will take longer than an ordinary feature review.

I will validate these areas in order and report them incrementally rather than bundle everything into one pass. There will likely be multiple request-changes rounds as each seam is exercised end to end; that is expected for a host integration and is not a judgment on the overall direction of the contribution. I am starting with installation and scheduler creation/wake-up behavior first.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The install guide can complete and register \memU\memu-bridging-pi, but the registered Windows task cannot launch Pi, so the record bridge never starts.

  • schedule install resolved Pi to the npm shim ...\pi.CMD and generated & '...\pi.CMD' -p $prompt.
  • Triggering the real S4U task reached the wrapper and read the prompt, then failed on that invocation with NativeCommandError: 系统找不到指定的文件 / LastTaskResult = 1.
  • bridge.log stayed empty, no Pi session was created, and the manifest/jobs did not advance.
  • schedule verify still reported success because it only checks registration plus an auth probe in the current interactive shell; it does not exercise the S4U task.
  • In the same S4U task, replacing only the invocation with the shim's node.exe <pi cli.js> -p $prompt immediately created a new Pi session and advanced the pending manifest/jobs. This isolates the failure to launching the npm .CMD shim under S4U, not Pi authentication or the prompt.

Please make the Windows scheduler invocation handle .cmd/.bat agent launchers correctly (and add coverage for that generated wrapper path), then verify by triggering the actual Task Scheduler entry and checking for a new Pi session plus advanced bridge traces. I am intentionally stopping this review pass at the installation/scheduler seam; the remaining host-adapter seams will be reviewed separately afterward.

@xnne-bot

xnne-bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

I tested the Windows install through the real agent-facing path:

  1. Removed the existing Pi memU task.
  2. Started a fresh Pi session and gave it only the rendered memu-pi docs install guide.
  3. Let Pi complete the installation without manual corrections.
  4. Confirmed \memU\memu-bridging-pi, its wrapper/prompt files, and retrieval artifacts were created.
  5. Triggered the actual S4U Task Scheduler entry and checked Pi sessions, bridge.log, manifests, and jobs.

The installation completed, and both schedule status and schedule verify reported success, but the real scheduled run failed:

LastTaskResult: 1
System.Management.Automation.RemoteException
NativeCommandError: 系统找不到指定的文件。

Content-free wrapper markers showed that Task Scheduler successfully started PowerShell and read bridge-prompt.txt; it failed exactly here:

& '...\pi.CMD' -p $prompt

bridge.log stayed empty, no new Pi session was created, and the manifest/jobs did not advance.

On this machine, Pi is installed through npm. Interactive PowerShell resolves pi to pi.ps1, while the scheduler installer uses shutil.which("pi") and freezes the npm pi.CMD shim into the wrapper. Claude Code works here because its scheduler path is a standalone claude.exe, not a batch shim.

As an isolation test, I changed only the same S4U task invocation to the executable and JS entry point used by the shim:

& '...\node.exe' '...\node_modules\@earendil-works\pi-coding-agent\dist\bundle\cli.js' -p $prompt

That immediately created a new Pi session and advanced the pending manifest and jobs. This confirms that S4U, Pi authentication, the model, and the prompt all work; the blocker is launching the npm .CMD shim.

Please make the Windows scheduler handle .cmd / .bat launchers correctly (or resolve the npm shim to its real executable entry point), preserve the prompt as one argument, and add wrapper coverage for that path. schedule verify should not be treated as end-to-end proof because its auth probe runs in the interactive process rather than the S4U task.

Since your development machine is macOS, please also run the equivalent black-box test there: give a fresh Pi session only memu-pi docs install, inspect the generated cron/launchd registration, trigger the real OS scheduler entry, and verify that bridge.log, a new Pi session, and the manifest/jobs all advance. A manual pi -p or manual wrapper run is not sufficient evidence that cron/launchd can wake the bridge with its non-interactive environment.

@wutongyuonce

Copy link
Copy Markdown
Contributor Author

Addressed the installation/scheduler review in commit 0da3459.

Windows npm launcher

  • When shutil.which() resolves an npm launcher to .cmd / .bat, the scheduler now uses npm's sibling .ps1 shim for the generated PowerShell wrapper. This avoids sending the prompt through cmd.exe; the wrapper invocation remains & '<shim>.ps1' -p $prompt, so the prompt stays one PowerShell argument.
  • Installation now refuses a batch launcher with no PowerShell companion instead of registering a task that cannot wake under S4U.
  • Added coverage through the complete install -> generated wrapper path for an npm pi.CMD, plus the missing-companion failure path.

schedule verify

  • Changed the install/verify output and the Windows scheduler guides to say explicitly that this checks registration plus a current-process headless-auth probe only.
  • Successful verify output now says that it did not run the S4U task and requires evidence from bridge.log, a new host session, jobs, and the promoted manifest.

macOS black-box verification

I ran a fresh Pi session with existing context files, skills, extensions, and prompt templates disabled, and supplied only the rendered memu-pi docs install guide. Before the run there was no Pi host directory and no crontab. The agent completed configuration, created the wrapper/prompt and hourly cron entry, installed retrieval, confirmed a fresh Pi session loaded the managed global context, and reported the install successfully.

I then temporarily moved only the generated cron entry to the next minute and let the macOS cron daemon trigger it (not a manual pi -p or wrapper invocation):

  • bridge.log: absent -> created -> 777 bytes after exit;
  • Pi session count: 53 -> 54 at cron wake-up;
  • jobs: 21 leftovers -> 7 newly prepared -> 0 after the final commit;
  • pending manifest: 2037 bytes -> 2453 bytes -> removed after promotion;
  • promoted manifest: absent -> 2037 bytes after the leftovers commit -> 2453 bytes after the final commit;
  • .bridge.lock: present while the cron run was active, absent after clean exit;
  • the generated cron entry was restored to its documented hourly 0 * * * * cadence.

The scheduled Pi run reported 28 jobs completed and both commits succeeded.

Validation:

  • Windows scheduler tests: 69 passed
  • make check: passed
  • full suite outside the listener-restricted sandbox: 607 passed, 8 skipped
  • git diff --check: passed

@wutongyuonce

Copy link
Copy Markdown
Contributor Author

Follow-up scope cleanup in c2af7cb: reverted the incidental Claude Code, Cursor, and Hermes BRIDGING_TASK.md edits. The review response is now limited to the shared Windows scheduler implementation/tests and Pi's scheduler guide. Windows scheduler tests remain 69 passed; pre-commit passes.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two blocking inline changes: keep the Pi host contract at the CLI boundary, and keep its installer outside memU's guide.

Comment thread src/memu/hosts/scheduling/windows.py Outdated
Comment thread src/memu/hosts/pi/INSTALL.md Outdated
Comment thread src/memu/hosts/pi/INSTALL.md Outdated
@wutongyuonce

Copy link
Copy Markdown
Contributor Author

Addressed the three blocking changes in 85aee52.

  • The generated Windows wrapper now invokes the host CLI name (pi) while retaining the resolved path for preflight/auth and for prepending its directory to the S4U task PATH.
  • Removed _resolve_scheduled_agent, the .ps1-specific install/verify guards, and their implementation-coupled tests. The replacement install -> generated wrapper regression starts from a resolved pi.CMD and asserts & 'pi' -p $prompt, the PATH prefix, and no frozen .CMD/.ps1 path.
  • Removed Pi installation instructions and the separate pi -p authentication probe from Pi's memU install guide.

Validation: Windows scheduler tests 68 passed; make check passed; full suite 606 passed, 8 skipped; git diff --check passed.

I cannot rerun the requested real Windows S4U black-box from this macOS host, so I am not claiming that runtime evidence. The generated-wrapper regression covers the requested CLI-boundary behavior; the real Task Scheduler wake-up still needs confirmation on Windows via the Pi session, bridge.log, jobs, and manifest traces.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CLI-boundary changes now match the requested design. We will run the real Windows S4U task on our side; you only need to keep the PATH-resolvable pi invocation supported on Windows. Please continue to ensure the macOS path works normally, including a real OS-scheduled wake-up that produces a new Pi session and advances bridge.log, jobs, and manifest traces. One remaining cleanup is inline.

Comment thread src/memu/hosts/pi/cli.py Outdated

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pi can now be launched by the Windows S4U task, but scheduled-session self-skip is broken in platform-independent transcript identity code and therefore affects Windows Task Scheduler as well as macOS/Linux cron or launchd. Windows is the concrete reproduction environment; the blocking mismatch is inline.

Comment thread src/memu/hosts/pi/sessions.py
@wutongyuonce

Copy link
Copy Markdown
Contributor Author

Addressed the Pi self-session identity mismatch in 4a3a102.

  • PiTranscriptSource.session_id() now removes Pi's timestamp prefix and returns the UUID exported through PI_SESSION_ID.
  • Added a source regression using the exact filename shape from the report.
  • Added a Pi-specific _cmd_prepare regression with a previously recorded bridge UUID and the marked current UUID. Both timestamp-prefixed transcripts are skipped, and the run writes neither transcript slices nor job files.

Validation: focused host/self-session tests 73 passed; make check passed; full suite 608 passed, 8 skipped; git diff --check passed.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The self-session identity fix is correct. One final blocking prepare-output privacy seam remains for Pi, inline.

Comment thread src/memu/hosts/pi/sessions.py
@wutongyuonce

Copy link
Copy Markdown
Contributor Author

Addressed the Pi prepare-output privacy seam in efc80cf, after rebasing the branch onto current main (54f9468).

  • Added a Pi-specific delete-only sanitize() at the shared prepare-output boundary.
  • Removed record id, parentId, and timestamp; message api, provider, model, usage, stopReason, rawStopReason, responseId, timestamp, and errorMessage; tool-result details; and content-block thinkingSignature.
  • Preserved conversation/tool content and unknown record, message, and content-block fields.
  • Added field-level coverage plus a prepare_transcripts integration regression proving prepared conversation/full output is sanitized while the source JSONL remains byte-for-byte unchanged and the pending cursor retains the original line count and timestamp.

Validation: focused host/self-session/scheduler tests 149 passed; make check passed; full suite 646 passed, 8 skipped; git diff --check passed.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for the current revision. I will conduct a further manual review later; it may take some time. Thank you for your patience.

@xnne-bot xnne-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

请移除 Pi 的 headless-auth gate,并明确本 adapter 只处理默认 Pi session store。

  1. src/memu/hosts/pi/cli.py 删除:

    needs_headless_auth=True,
    auth_hint=...,

    安装是由已运行的 Pi agent CLI 执行的;能到达此指南即表示该 Pi CLI 已有可工作的 provider 配置。Claude Code 的问题是 Desktop 登录与 standalone claude CLI 是两个身份边界,Pi 没有这个前提。memu-pi schedule install / verify 不应再执行 pi -p ping,或将 /login、API key、persistent credential 作为 memU 的安装门。

  2. memu-pi 暂时只支持默认、固定的 Pi store:~/.pi/agent/sessions。删除 sessions.pyPI_CODING_AGENT_DIR / PI_CODING_AGENT_SESSION_DIR 的环境读取,删除 Pi spec 的 schedule_prepare_session_dir=True,并删除 INSTALL/BRIDGING_TASK 中关于 PI_CODING_AGENT_DIRPI_CODING_AGENT_SESSION_DIRsettings.json.sessionDir、per-run --session-dir 以及改写 scheduled prompt 的支持承诺。

    这是有意的拒绝,不是尚未补齐的 wrapper 功能。bridging 的 session source、cursor/manifest 和 host working tree 必须长期绑定同一个稳定 store;而这些 Pi override 可在安装后随时改为另一个目录、unset,或只存在于交互 shell,cron/S4U 的新进程无法可靠知道以后每次 Pi 使用哪一个。把安装时路径写死会在变更后静默读取旧 store;每次跟随环境会使同一个 cursor 在不同 store 间切换,造成漏读、重读或错误 self-skip。转运变量到 wrapper 或在 prompt 烘焙路径都只是隐藏这个不稳定契约。

    因此,用户将 Pi 配置到非默认 agent/session directory 时,该目录的 sessions 不会进入 memU 的 memorize/bridging 流程;这比声称支持、却可能在未来的 scheduler run 中读错目录安全。未来若需要支持迁移,必须先设计显式、持久化、可验证的 source migration,而不是把每次运行可变的 Pi 配置当作 scheduled bridge 的数据源。

  3. 删除 INSTALL/BRIDGING_TASK 中 scheduler “runs a headless authentication probe”及“checks registration and authentication”的表述。验证应是注册 task 后触发真实 OS task,确认新 Pi session、bridge.log 和 jobs/manifest traces。

@NevaMind-AI NevaMind-AI deleted a comment from xnne-bot Sep 4, 2026
@NevaMind-AI NevaMind-AI deleted a comment from xnne-bot Sep 4, 2026
Pi's interactive CLI and scheduled pi -p share ~/.pi/agent/auth.json, so memU must not probe /login or an API key at schedule install/verify. That gate belongs to Claude Code, whose desktop login is invisible to standalone claude. Scheduled prepare stays on the default ~/.pi/agent/sessions layout; baking an install-time --session-dir would freeze a path the OS task cannot see.
@wutongyuonce

wutongyuonce commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for the careful review.

Addressed the scheduler-auth and custom-session-dir review in 8376fb0.

This is not the Claude Code case. Claude Code needs needs_headless_auth=True because the Desktop login is invisible to standalone claude -p, so the S4U task really does need a persistent CLI credential (setup-token / ANTHROPIC_API_KEY). Interactive pi and scheduled pi -p are the same CLI and share ~/.pi/agent/auth.json; memU should not treat /login, an API key, or a headless-auth probe as an install gate.

  • Removed needs_headless_auth=True and auth_hint from the Pi spec, so schedule install / verify no longer run pi -p ping.
  • Removed schedule_prepare_session_dir=True. Scheduled prepare stays memu-pi prepare on the default ~/.pi/agent/sessions layout; install-time PI_CODING_AGENT_DIR / PI_CODING_AGENT_SESSION_DIR / settings.json paths are not baked in, and the wrapper is not extended to forward them.
  • INSTALL.md / BRIDGING_TASK.md no longer mention a headless authentication probe, custom session directories, or rewriting the scheduled prompt. Verify is task registration plus a real OS run that produces a new Pi session, bridge.log, and jobs/manifest traces.

Validation: focused host/self-session/scheduler tests 185 passed; git diff --check passed.

from typing import ClassVar

from memu.hosts.base import RecordKind, TranscriptSource

@xnne-bot xnne-bot Sep 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] 请把这两个值收窄为固定默认常量:

AGENT_DIR = "~/.pi/agent"
SESSION_DIR = "~/.pi/agent/sessions"

并移除 os import。

这不是遗漏 scheduler forwarding 的请求,而是有意拒绝 Pi 的可变 directory override。当前文档和 scheduler 已声明 scheduled bridge 只支持默认 layout,也已不再把安装时 session dir 烘焙到 prompt;但此处仍会读取 PI_CODING_AGENT_DIR / PI_CODING_AGENT_SESSION_DIR,且 cli.pyAGENT_DIR 构造 instruction/skills path。

若安装时的 Pi 进程设置了这些变量,手动 memu-pi prepare 会以非默认 store 产生 cursor/manifest,而之后 cron/S4U 的新环境通常回到默认 store;反过来,用户之后修改或 unset 变量也会令同一 ~/.memu/hosts/pi working tree 在不同 session source 间切换。写死安装时的路径、把变量转运到 wrapper、或把它们写进 prompt,都只会把同一不稳定契约延后暴露。

默认 ~/.pi/agent/sessions 是本 adapter 当前唯一稳定 source。用户将 Pi 配置到非默认 agent/session directory 时,那个 directory 的 sessions 应明确不进入 memU memorize/bridging,优于声称支持却可能在后续 scheduled run 静默读错 store。未来若要支持它,需要单独的显式、持久化、可验证 source migration;本 PR 不要实现该能力。

AGENT_DIR and SESSION_DIR no longer follow PI_CODING_AGENT_DIR or PI_CODING_AGENT_SESSION_DIR. Manual prepare and the later cron/S4U prepare must share ~/.pi/agent/sessions; reading process env would make the install-time cursor track a store the scheduled task does not inherit. Custom Pi directories stay out of this adapter; a later PR would need an explicit source migration.
@wutongyuonce

wutongyuonce commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

@xnne-bot Thank you for the careful follow-up.

Addressed the remaining directory-override leak in 5576bad.

8376fb0 stopped baking an install-time path into the scheduled prompt, but sessions.py still read PI_CODING_AGENT_DIR / PI_CODING_AGENT_SESSION_DIR on every memu-pi start. A manual prepare in an env that had those variables would write the cursor against a custom store; the later cron/S4U prepare usually has a clean env and would read ~/.pi/agent/sessions. Same working tree, two sources.

  • AGENT_DIR / SESSION_DIR are now the fixed defaults ~/.pi/agent and ~/.pi/agent/sessions. The os import is gone.
  • Instruction and skills paths follow that same default through cli.py.
  • Custom Pi agent/session directories are out of this adapter: those sessions do not enter memorize/bridging. An explicit, persistent source migration is a later PR, not this one.

Validation: focused host/self-session/scheduler tests 186 passed; git diff --check passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] add a dedicated pi coding agent adapter

2 participants